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Summary 

Between 2000 and 2009, the Hispanic population more than doubled in 25 of 67 Pennsylvania 
counties. Over the same period, the Hispanic student population in Pennsylvania schools also 
rose, from 4 percent to 8 percent (Pennsylvania State Data Center 2011). The focus on His- 
panic students’ level of academic achievement rose along with this rapid population growth. 

Recent research reveals that, although the achievement gap between ethnic subgroups at 
the national level has been shrinking over the past five years, the gap remains wide (Aud et al. 
2010). This is also the case and a matter of concern in Pennsylvania. 

Research has identified several student-level factors associated with academic achievement 
among ethnic minority students, including gender and socioeconomic status (Freeman 2004; 
McGraw, Lubienski, and Strutchens 2006; Pong 2010); English language learner status (Eamon 
2005; Reardon and Galindo 2007; Terwilliger and Magnuson 2005); special education status 
(Sanchez et al. 2009); and mobility (Suarez-Orozco, Gaytan, and Kim 2010). School-level fac- 
tors also influence student achievement. One contributing factor is the proportion of special 
needs students in a school, whether because they come from a low-income household (Sirin 
2005), have a disability (Kalambouka et al. 2007), or are English language learner students 
(Schmid 2001). School-level factors also include school dropout rates (Sanchez et al. 2009), 
school size (Crosnoe 2005), the proportion of ethnic minority students (Coleman 1966), stu- 
dent-teacher ratio (Nye, Hedges, and Konstantopoulos 2004), and school locale (Pong 1998). 

Two research questions guided this study: 

• How does the performance of Pennsylvania grade 8 Hispanic students on the 
2007/08 Pennsylvania System of School Assessment English language arts and 
math tests compare with that of grade 8 non-Hispanic White, Black, and other non- 
Hispanic students? 

• Among Pennsylvania grade 8 Hispanic students, which student- and school-level 
characteristics are associated with performance on the 2007/08 Pennsylvania System 
of School Assessment English language arts and math tests? 

The data used for this study were the Pennsylvania System of School Assessment (PSSA) 
scores collected by the Data Recognition Corporation for the Pennsylvania Department of 
Education. In addition to the PSSA, the Pennsylvania Department of Education’s Bureau 
of Assessment and Accountability provided demographic data for all grade 8 students in 
2007/08, and publicly available school-level data were also accessed from the department’s 
website. Additional demographic school data were obtained from the publicly accessible 
Common Core of Data of the National Center for Education Statistics (U.S. Department of 
Education 2008). 

Key findings show: 

• The difference in performance of grade 8 Hispanic students and non-Hispanic stu- 
dents was 174 scaled score points on the PSSA reading test and 123 scaled score 
points on the math test. 

• Scores on both the PSSA reading and math tests were significantly lower for Hispanic 
students than for White students and for students of other ethnicities. Hispanic 
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students’ and Black students’ scores were not significantly different on the reading 
test, but scores on the math test were significantly higher for Hispanic students than 
for Black students. 

• There was a statistically significant relationship between Hispanic students’ PSSA 
test scores and gender, special education status, eligibility for free or reduced-price 
lunch status,' and English language learner status. There was not a statistically sig- 
nificant relationship between PSSA scores and migrant status. 

• There was a statistically significant relationship between Hispanic students’ PSSA 
test scores and school size, the proportion of Hispanic students eligible for free or 
reduced-price lunch, and whether the school reported having students who dropped 
out.^ There was not a statistically significant relationship between PSSA scores and 
the percentage of grade 8 English language learner students, percentage of grade 
8 students receiving special education services, percentage of Hispanic students, 
student-teacher ratio, and school locale (urban, suburban, town, or rural). 

April 2012 
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Why this brief? 

Between 2000 and 2009, the Hispanic popu- 
lation more than doubled in 25 of Pennsylva- 
nia’s 67 counties, and the Hispanic student 
population in Pennsylvania schools rose from 
4 percent to 8 percent (Pennsylvania State 
Data Center 2011). With this rapid population 
growth, the focus on Hispanic students’ level 
of academic achievement has also increased. 
Although the achievement gap between 
ethnic subgroups has been shrinking at the 
national level over the past five years, the gap 
remains wide nationally (Aud et al. 2010). 
This is also the case and a matter of concern in 
Pennsylvania. 

To learn more about this gap in Pennsyl- 
vania, this brief compares Pennsylvania grade 
8 Hispanic students and their non-Hispanic 
counterparts on state tests of English lan- 
guage arts and math. It examines the associa- 
tion between Hispanic students’ test perfor- 
mance and several student- and school-level 
characteristics. 

National trends 

The Hispanic population is the largest racial/ 
ethnic minority group in the United States, 
accounting for more than half the country’s 
population growth over 2000-10 (Humes, 
Jones, and Ramirez 2011). Despite making up 
about 20 percent of public school K-12 students 
nationwide (Suarez-Orozco, Gaytan, and Kim 
2010), Hispanic students are also the most edu- 
cationally disadvantaged racial/ethnic group — 
by a number of measures (Schneider, Martinez, 
and Owens 2006). An achievement gap per- 
sists between Hispanic students and their non- 
Hispanic peers throughout middle and high 
school. Further, Hispanic youth are less likely 
than either White or Black youth to feel a sense 
of belonging in school (Schneider et al. 2006). 

Although the math proficiency of Hispanic 
students as a group is similar to that of Black 



students in elementary school, Mexican-ori- 
gin students (the largest Hispanic subgroup) 
have lower math proficiency scores in elemen- 
tary grades than any other racial/ethnic sub- 
group, including Black students (Reardon and 
Galindo 2007). Hispanic high school students 
take fewer advanced math courses than their 
White counterparts and fewer advanced sci- 
ence courses than either White or Black stu- 
dents (Schneider, Martinez, and Owen 2006). 

About 33 percent of Hispanic children who 
enter kindergarten do so with two or more risk 
factors, such as low maternal education, single- 
parent household, welfare dependency, and 
parental low proficiency in English (Schneider, 
Martinez, and Owens 2006). The percentage 
of Black children facing two or more risk fac- 
tors is 6 percentage points lower, at 27 percent. 
Hispanic students have also been reported to 
have higher dropout rates and lower school 
completion rates than any other racial/ethnic 
subgroup (Planty et al. 2008). 

In light of these findings, it is important 
that policymakers understand individual and 
school factors in Hispanic students’ academic 
performance (Miguel and Donato 2010). 

Pennsylvania trends 

Mirroring the rise in the Hispanic population 
across the United States, Pennsylvania saw a 
64 percent increase in the Hispanic popula- 
tion over 2000-09 (Pennsylvania State Data 
Center 2011). The 2010 census data recorded 
Hispanics as the fastest growing racial/ethnic 
minority group in the state (Pennsylvania State 
Data Center 2011). Of 67 Pennsylvania coun- 
ties, 25 (37 percent) saw their Hispanic popu- 
lation more than double over 2000-09 (Penn- 
sylvania State Data Center 2011). At the same 
time, the non-Hispanic population across the 
state declined 1.4 percent. 

In 2009/10, Hispanic students made 
up 8 percent of Pennsylvania’s public school 
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population, twice the 4 percent in 1999/00 
(Pennsylvania State Data Center 2000; U.S. 
Department of Education 2011). This demo- 
graphic shift has become a key focus in the 
education and political discourse. On Decem- 
ber 3, 2009, the Pennsylvania Governor’s Advi- 
sory Commission on Latino Affairs and the 
Pennsylvania Department of Education held a 
Latino Students Educational Excellence Sum- 
mit. Teachers, administrators, schools of higher 
education, businesses, and nonprofit organiza- 
tions were represented, and the conversation 
focused on the increasing number of Hispanic 
students entering Pennsylvania schools and the 
achievement gap between Hispanic students 
and other racial/ethnic subgroups. The Penn- 
sylvania Department of State Secretary of the 
Commonwealth talked about the need for bet- 
ter understanding of Hispanic student achieve- 
ment and pointed to the Obama administra- 
tion’s focus on Hispanic student achievement. 
This emphasis was later reinforced nationally 
when President Obama signed the “White 
House Initiative on Educational Excellence 
For Hispanics” on October 19, 2010 (Executive 
Order 13,555 2010). 

The achievement of Hispanic students is 
increasingly a concern for educators and deci- 
sionmakers in Pennsylvania, particularly as 
the Hispanic student population is growing. 
The 2009 National Assessment of Educational 
Progress (NAEP) data indicated a 29 percent- 
age point gap between White and Hispanic 
grade 8 students in Pennsylvania in reading 
performance and a 28 percentage point gap 
between these groups in math performance 
(NAEP 2009a,b).^ (See appendix A for a more 
comprehensive discussion of the literature.) 

To learn more about this gap, this report 
compares the performance of grade 8 Hispanic 
students and their non-Hispanic counterparts 
on the Pennsylvania System of School Assess- 
ment (PSSA) tests of English language arts and 
math (see box 1 for definition of key terms). The 
PSSA is a standards-based, criterion-referenced 
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assessment that measures a student’s attain- 
ment of academic standards. Every Pennsylva- 
nia student in grades 3-8 and grade 11, unless 
excused as described below, is assessed in Eng- 
lish language arts (reading) and math (Data 
Recognition Corporation 2009).^ Scores reveal 
the performance of the population of Penn- 
sylvania public school students and offer addi- 
tional insight on performance when compared 
with the NAEP scores, which are based on stu- 
dents from a sample of available schools. PSSA 
scores have been found to be reliable and valid 
(Thacker 2004). 

Students may be excused from participat- 
ing in the PSSA for several reasons: by paren- 
tal request, if they are assessed through alter- 
nate means, if they experienced an extended 
absence, if they should have but have not yet 
received an individualized education program, 
if they have been identified as an English lan- 
guage learner (ELL) student and are in their 
first year of enrollment in the United States, or 
if they have a medical emergency. 

Research questions 

Two research questions guide this study: 

• How does the performance of Penn- 
sylvania grade 8 Hispanic students on 
the 2007/08 Pennsylvania System of 
School Assessment English language 
arts and math tests compare with that 
of grade 8 non-Hispanic White, Black, 
and other non-Hispanic students? 

• Among Pennsylvania grade 8 Hispanic 
students, which student- and school- 
level characteristics are associated with 
performance on the 2007/08 Penn- 
sylvania System of School Assessment 
English language arts and math tests? 

This study focuses on grade 8 for two rea- 
sons. First, national and state attention on His- 
panic student academic performance focuses 
on middle school, a pivotal time for school 
attachment and intervention for lowering the 
high school dropout rate (LeCroy and Krysik 
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BOX 1 

Key terms 

Dropout. A student who, for any rea- 
son except death, leaves school before 
graduation without transferring to 
another school or institution. School 
districts provide dropout data to the 
Pennsylvania Department of Educa- 
tion. This study treats “school without 
dropout” as a discrete dichotomous 
variable and thus does not account for 
differences in dropout rates. 

English language learner student. Ac- 
cording to the Pennsylvania Depart- 
ment of Education, a student whose 
dominant language is not English. 
The designation is based on informa- 
tion received from a home language 
survey and the results of the Assess- 
ing Comprehension and Communi- 
cation in English State-to-State for 
English Language Learners Place- 
ment Test of the World-Class In- 
structional Design and Assessment. 

Ethnicity. In the PSSA records, eth- 
nicity is disaggregated into six catego- 
ries: American Indian or Alaskan Na- 
tive, Asian or Pacific Islander, Black/ 
African American non-Hispanic, La- 
tino/Hispanic, White non-Hispanic, 
and multi-racial/ethnic. This report 
uses the term “ethnicity” rather than 
“race/ethnicity” because that is the 
term used by education leaders and 
policymakers in Pennsylvania, the 
primary audience for this report. 

Eree or reduced-price lunch eligibility. A 
proxy for low-income status. Children 
from a household with an income at or 
below 130 percent of the poverty level 
are eligible for free meals. Children 
from a household receiving Temporary 



Assistance for Needy Eamilies or 
food stamp benefits automatically 
qualify for free meals. Children from 
a household whose income is 130-185 
percent of the poverty level are eligible 
for reduced-price meals (Pennsylvania 
Department of Education n.d. b). 

Hispanic student. As defined on 
Pennsylvania’s 2007/08 standard- 
ized tests, all students whose parents 
came from Cuba, the Dominican 
Republic, Mexico, Puerto Rico, Cen- 
tral or South America, or any other 
country once colonized by Spain. 
This variable was self-reported. 

Migrant. “Any child domiciled 
temporarily in any school district for 
the purpose of seasonal or temporary 
employment, but not acquiring resi- 
dence therein, and any child accom- 
panying his parent or guardian who is 
so domiciled” (Pennsylvania Depart- 
ment of Education 2008, p. 6). School 
personnel make this designation. 

Multilevel modeling. A statistical 
technique that accounts for the 
nested structure of data, as when stu- 
dents are grouped into larger units, 
such as the classroom or school. 

Multiple imputation. A statistical 
technique that replaces missing val- 
ues through simulation. Each com- 
plete dataset with simulated values 
is analyzed using standard methods, 
and the results are combined to 
produce estimates that incorporate 
missing-data uncertainty. 

Pennsylvania System of School Assess- 
ment (PSSA). The annual standards- 
based, criterion-referenced assessment 
used to measure a student’s attainment 



of academic standards. Each student in 
grades 3-8 and 11 is assessed in read- 
ing and math, each student in grades 
5, 8, and 11 is assessed in writing, and 
each student in grades 4, 8, and 1 1 
is assessed in science (Pennsylvania 
Department of Education 2008). 

School-level characteristics. School size, 
percentage of grade 8 English lan- 
guage learner students, percentage of 
grade 8 students receiving special edu- 
cation services, percentage of students 
receiving free or reduced-price lunch, 
student-teacher ratio, dropout status, 
and school locale (urban, suburban, 
town, or rural). These variables, 
defined in appendix E, are from the 
Common Core of Data (U.S. Depart- 
ment of Education 2008). 

Special education. A student with an 
individualized education program 
(not including gifted programs). 

Standard deviation. A measure of 
variability quantifying the average 
distance of a set of scores from the 
mean of the scores. The PSSA reports 
scores in standard deviation units. 

Standard error. A statistical term 
representing how accurately a sample 
represents a population. The mean 
for a sample drawn from a popula- 
tion will be different from the mean 
for the whole population. This differ- 
ence is the standard error. 

Student-level characteristics. Gender, 
special education status, eligibil- 
ity for free or reduced-price lunch, 
English language learner status, and 
migrant status. These variables, de- 
fined in appendix E, were obtained 
from the PSSA dataset. 
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2008). Second, Pennsylvania policymakers 
have noted the importance of intervening with 
Hispanic students in middle school for increas- 
ing academic achievement and graduation 
rates. 

The PSSA dataset was provided to the 
researchers by the Pennsylvania Department 
of Education with no student identifiers to 
protect student confidentiality. The data from 
the PSSA and the Common Core of Data (U.S. 
Department of Education 2008) for 2007/08 
were the most complete data at the time of 
this study and were chosen for that reason. As 
newer datasets based on the same assessments 
become available to the Pennsylvania Depart- 
ment of Education, this study can be updated 
using the same methodology (see box 2 and 
appendixes B-D for a description of the data 
sources and study methods). 



Comparison of performance of 
Hispanic and non-Hispanic students 
on the 2007/08 grade 8 Pennsylvania 
System of School Assessment English 
language arts and math tests 
The performance of grade 8 Hispanic students in 
Pennsylvania, compared with their non-Hispanic 
counterparts, was examined using the English 
language arts (reading) and math scaled scores 
from the 2007/08 PSSA. Non-Hispanic students 
were first analyzed as a single group and then sepa- 
rately as non-Hispanic White students (“White”), 
non-Hispanic Black students (“Black”), and other 
non-Hispanic students (“other”).^ 

Comparisons of characteristics of Hispanic 
and non-Hispanic student populations 

To establish a context for the discussion 
on differences in test performance, several 



BOX 2 

Data and methods 

The data for this study come from 

three sources: 

• The Pennsylvania System of 
School Assessment (PSSA) data, 
provided by the Pennsylvania 
Department of Education, show 
students’ test performance and 
demographic characteristics for 
2007/08. The PSSA data cover 
the population of grade 8 stu- 
dents from all of Pennsylvania’s 
regular public schools, including 
charter schools. 

• Pennsylvania public school 
data were extracted from the 
2007/08 Common Core of Data 
(U.S. Department of Education 
2008). The data contain unique 
state school identification num- 
bers that the study could link to 
the PSSA. 



• School dropout data for 

2007/08 came from the Pennsyl- 
vania Department of Education 
website (Pennsylvania Depart- 
ment of Education n.d. a). The 
study linked the dropout data to 
the PSSA through school names. 

Descriptive analyses were conducted 
to compare the performance of 
grade 8 Hispanic students on the 
English language arts and math tests 
with that of grade 8 non-Hispanic 
students as a group and separately as 
White students, Black students, and 
students of other ethnicities. 

Among the PSSA grade 8 students, 
2,952 (2 percent) were missing 
English language arts test scores, 
and 1.8 percent (2,609) were missing 
math test scores. Also, 204 students 
(less than 1 percent) were missing 
information on gender. After students 



with missing data were removed, the 
descriptive analyses were based on 
140,967 grade 8 students (98 percent). 
The removal of such a small percent- 
age of students was not expected to 
bias the results (appendix B). 

The Common Core of Data dataset, 
however, was missing a substantial 
number of values (33 percent). Mul- 
tiple imputation was used to address 
missing data (appendix C), and 
sensitivity analyses were conducted 
to test the effect of the missing data 
procedures on study outcomes (ap- 
pendix D). 

Multilevel modeling was applied to 
examine the associations between 
student- and school-level characteris- 
tics and Hispanic students’ perfor- 
mance on the PSSA. (See appendix E 
for more details on the methodology, 
including the multilevel equations.) 
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comparisons were made between the Hispanic 
and non-Hispanic student populations 
(table 1). In 2007/08, 9,040 grade 8 students 
in Pennsylvania were Hispanic (6.4 percent of 
the total grade 8 population of 140,967). Of 
the remaining 131,927 non-Hispanic grade 8 
students, 80 percent (105,644) were White, 
17 percent (21,869) were Black, and 3 percent 
(4,414) were American Indian/Alaskan Native, 
Asian/ Pacific Islander, or multiracial (grouped 
as “other”). 

Descriptive statistics in table 1 show simi- 
lar gender composition within and between 
groups but noticeable differences in several 
student variables between Hispanic and non- 
Hispanic grade 8 students (these differences 
are significant at thep = 0.001 level using a chi- 
square test): 

• Hispanic students were more likely 
to be eligible for free or reduced-price 
lunch (71.8 percent) than were non- 
Hispanic students (30.3 percent). 

• Hispanic students were more likely to 
receive special education services (16.9 
percent) than were non-Hispanic stu- 
dents (14.6 percent). 

• In 2007/08, 17.6 percent of Hispanic 
grade 8 students were current ELL 
students, compared with 0.7 percent 
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of non-Hispanic grade 8 students. Just 
1.3 percent of non-Hispanic students 
were current or former ELL students, 
compared with 30.0 percent of His- 
panic students. 

• Migrant students constituted a small 
share of the total grade 8 population 
in Pennsylvania, and most migrant 
students were Hispanic (2.8 percent of 
Hispanic students). 

Hispanic and non-Hispanic grade 8 stu- 
dents differed in the characteristics of the 
schools they attended, and in many cases 
the differences were statistically significant 
(table 2). Hispanic students attended schools 
with significantly more peers receiving special 
education services, significantly more English 
language learner students, and significantly 
more students of their own ethnicity than did 
non-Hispanic students. Hispanic students’ 
schools had a significantly higher percentage 
of students eligible for free or reduced-price 
lunch, and were significantly more likely to be 
located in cities than were schools attended by 
non-Hispanic students. However, Hispanic 
students attended schools similar to those 
attended by non-Hispanic students in student- 
teacher ratio, school size, and whether the 
school had dropouts. 



TABLE 1 

Characteristics of grade 8 Hispanic, White, Black, and other ethnicity students in Pennsylvania, 2007/08 



Student variable 


Hispanic 
(n = 9,040) 


White 

(n = 105,644) 


Black 

(n = 21,869) 


Other 
(n = 4,414) 


All non-Hispanic 
(n = 131,927) 


Male 


50.2 


51.4* 


50.2 


50.8 


51.2 


Eligible for free or reduced-price lunch 


71.8 


22.0*** 


69.2 


34.8*** 


30.3*** 


Special education 


16.9 


14.2*** 


18.0 


6.6*** 


14.6*** 


Current English language learner 


17.6 


0.3*** 


1.0*** 


10.1*** 


0.7*** 


Former English language learner 


12.4 


0.3*** 


0.4*** 


10.9 


0.6*** 


Never an English language learner 


70.0 




98.6*** 


79.0*** 


98.6*** 


Migrant 


2.8 


0.01*** 


0.03*** 


0.4*** 


0.03*** 



* Significantly different from Hispanic students at p = 0.05;*** significantly different from Hispanic students at p = 0.001. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education. 



5 




REL Technical Brief REL 2012-No. 025 



Associations 



TABLE 2 



School characteristics for grade 8 Hispanic and non- 


-Hispanic students in Pennsylvania, 


2007/08 


School variable 


Hispanic 


Non-Hispanic 


Percentage of grade 8 students receiving special education 


17.3 


15.4* 


services 






Percentage of grade 8 English language learner students 


21.7 


2 2*** 


Percentage of grade 8 Hispanic students 


59.7 


4.1*** 


Percentage of students eligible for free or reduced-price 


80.7 


31.0*** 


Student-teacher ratio 


14.4 


14.3 


School size 


821.4 


787.3 


School with no dropouts 


0.90 


0.85 


Suburban school 


0.09 


0.49*** 


School in town 


0.01 


0.16*** 


Rural school 


0.02 


0.20*** 


Urban school 


0.88 


0.15*** 



* Significantly different from Hispanic students at p = 0.05; *** significantly different from Hispanic students at p = 0.001 . 

Wore; There were a total of 886 schools. The p-values were based on f-tests and show differences in school characteristics between Hispanic and non-Hispanic 
students. The school statistics for Hispanic and non-Hispanic students were weighted by the proportion of Hispanic and non-Hispanic students who attended 
the school to represent the characteristics of schools that average Hispanic students attended and those that average non-Hispanic students attended. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education. 



Test performance comparisons 

In 2007/08, Hispanic students in Pennsylva- 
nia had lower reading and math scores than 
did non-Hispanic students. The average scaled 
reading score for Hispanic students was 1,281, 
174 points lower than the average for non- 
Hispanic students (1,455; figure 1). 

Similarly, the average scaled math score for 
Hispanic students was 1,281, 123 points lower 
than the average for non-Hispanic students 
(1,404; figure 2). The differences in both the 
reading and math test score were statistically 
significant. 

When non-Hispanic students were disaggre- 
gated into Black students. White students, and 
other students (students of other race/ethnic- 
ity), Hispanic students still scored lower on both 
the reading and math tests than White students 
and other students (tables 3 and 4). For reading. 
White students and other students scored sig- 
nificantly higher than Hispanic students, while 
Black students did not score differently. For 
math. White students and other students scored 



significantly higher than Hispanic students. The 
Hispanic-White gap, to the disadvantage of His- 
panic students, was about 206 scaled score points 
in reading and about 150 scaled score points in 
math. On the reading test, Hispanic students 
performed similar to Black students, with an 11 
scaled score point difference in favor of Black stu- 
dents. On the math test, Hispanic students scored 
27 scaled score points higher than Black students. 

Associations between grade 8 
Hispanic student performance on 
the Pennsylvania System of School 
Assessment and student- and 
school-level characteristics 
The second research question examines 
whether student and school characteristics 
are associated with Hispanic student perfor- 
mance on the PSSA. Of the 886 schools stud- 
ied, 282 had no Hispanic students in grade 8, 
and 235 had only one or two. Five percent of 
these 886 schools had 50 or more Hispanic stu- 
dents in grade 8, while less than 1 percent had 
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FIGURE 1 

Average scaled reading scores on 
the Pennsylvania System of School 
Assessment for grade 8 Hispanic and 
non-Hispanic students, 2007/08 



FIGURE 2 

Average scaled math scores on the 
Pennsylvania System of School 
Assessment grade 8 Hispanic and 
non-Hispanic students, 2007/08 



Scaled score 




Scaled score 




Fllspanic Non- Black White Other 

Hispanic 



*** Significantly different from Hispanic students at 
p = .001. 

Note: Scaled scores range from 700-2,646 
(mean = 1,444; standard deviation = 247). 

Source.- Authors' analysis of data from the 2007/08 
Pennsylvania System of School Assessment provided 
by the Pennsylvania Department of Education. 



*** Significantly different from Hispanic students at 
p = .001. 

Note: Scores range from 700-2,259 (mean = 1,396; 
standard deviation = 221). 

Source: Authors' analysis of data from the 2007/08 
Pennsylvania System of School Assessment provided 
by the Pennsylvania Department of Education. 



TABLE 3 

Scores on the Pennsylvania System of School Assessment 2007/08 reading test for grade 8 students, by 
ethnicity 





Hispanic 


White 


Black 


Other 


All non-Hispanic 


Score 


(n = 9,040) 


(n = 105,644) 


(n = 21,869) 


(n = 4,414) 


(n = 131,927) 


Average score 


1,281.18 


1,487.47 


1,291.76 


1,483.61 


1,454.90 


Standard deviation of score 


241.68 


230.69 


233.05 


269.71 


243.60 


Differences from Hispanic students 


na 


206.29*** 


10.58 


202.43*** 


173.72*** 



*** Significantly different from Hispanic students at p = 0.001. 
na is not applicable. 

Wofe. The p-values were based on f-tests with adjustment for clustering within schools. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education. 



more than 200 (from 211 to 267). The data for 
answering this second research question were 
restricted to Hispanic students. 

Because the students are clustered (or 
nested) within these schools, the analysis 



method selected must be able to handle com- 
plex relationships within the data. Multilevel 
modeling was used because it accounts for the 
nested structure of the data. For more detail, see 
appendix E or Raudenbush and Bryk (2002). 
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TABLE 4 

Scores on the Pennsylvania System of School Assessment 2007/08 math test for grade 8 students, by ethnicity 



Score 


Hispanic 
(n = 9,040) 


White 

(n = 105,644) 


Black 

(n = 21,869) 


Other 
(n = 4,414) 


All non-Hispanic 
(n= 131,927) 


Average score 


1,281.13 


1,431.28 


1,253.87 


1,493.46 


1,403.96 


Standard deviation of score 


199.76 


210.91 


190.65 


249.25 


219.83 


Differences from Hispanic students 


na 


150.15*** 


-27.26*** 


212.33*** 


122.83*** 



*** Significantly different from Hispanic students at p = 0.001. 
na is not applicable. 

Note.' The p-values were based on f-tests with adjustment for clustering within schools. 

Source; Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education. 



Student characteristics included in the 
regression model were gender, eligibility for 
free or reduced-price lunch, special education 
participation, English language learner status, 
and migrant status. These characteristics were 
chosen because previous research had identified 
them as related to performance by racial/ethnic 
minority students (Freeman 2004; Galindo 
2007; McGraw, Lubienski, and Strutchens 
2006; Pong 2010; Reardon and Ream 2005; 
Sanchez et al. 2009; Suarez-Orozco, Gaytan, 
and Kim 2010), and they were available in the 
Pennsylvania Department of Education data- 
set. School characteristics were also selected 
based on a literature review,^ as well as avail- 
ability in the dataset. The model included the 
percentage of grade 8 students receiving spe- 
cial education services, percentage of grade 8 
students who were English language learner 
students, percentage of students who were 
Hispanic, percentage of students eligible for 
free or reduced-price lunch, student-teacher 
ratio, school size, school with no dropouts, and 
school location.^ 

Multilevel modeling was used to analyze 
two dependent variables separately: scaled 
reading test scores and scaled math test scores. 
A separate model was estimated for each test 
(table 5). The intercept in the model is the aver- 
age achievement of Hispanic students. 

When the regression coefficient for a vari- 
able was statistically significantly different 



from zero at the p = 0.05 level, the relation- 
ship between the variable and the test scores 
was considered to be statistically significant. In 
some cases, results are also reported for signifi- 
cance at thep = 0.001 level. 

Student-level variables 

There were statistically significant associations 
at thep = 0.001 level between grade 8 Hispanic 
students’ PSSA test scores and five of the six 
student-level variables (see table 5): 

• Hispanic female students were pre- 
dicted to score 31 scaled score points 
higher on the reading test than were 
Hispanic male students, but Hispanic 
male students were predicted to score 
25 points higher on the math test than 
were Hispanic female students. 

• Hispanic students eligible for free or 
reduced-price lunch were predicted to 
score 39 scaled score points lower on 
the reading test and 25 points lower on 
the math test than were non-eligible 
Hispanic students. 

• Hispanic students receiving special edu- 
cation services were predicted to score 
219 scaled score points lower on the 
reading test and 190 points lower on the 
math test than were Hispanic students 
not receiving special education services. 

• Current Hispanic ELL students were 
predicted to score 238 scaled score 
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TABLE 5 

Multilevel regression results of the association between student- and school-level characteristics and grade 8 
Hispanic students' scores on the Pennsylvania System of School Assessment reading and math tests, 2007/08 



Statistic and variable 


Reading 


Math 


Percentage of Hispanic students 


-1.63 

(3.21) 


-0.71 

(2.96) 


Percentage of students eligible 
for free or reduced-price lunch 


-17.07*** 

(2.41) 


-12.60*** 

(2.21) 


Student-teacher ratio 


0.77 

(1.46) 


0.37 

(1.32) 


School size 


-2.29 

(1.22) 


-2.83* 

(1.10) 


School with no dropouts 


24.74* 

(12.51) 


33.30 

(11.19) 


Suburban school 


-1.57 

(14.12) 


-2.55 

(12.69) 


School in town 


-10.28 

(18.02) 


-16.44 

(16.12) 


Rural school 


13.55 

(17.12) 


-3.09 

(15.63) 




Between-school variance 


102.60*** 


77.85*** 


Within-school variance 


221.37 


186.25 


Total variance 


323.98 


264.10 


Percentage of variance explained 
by the model 


24.37 


16.96 



Statistic and variable 


Reading 


Math 


Standard deviation of test score 


241.68 


199.76 


Intercept 


1,286.56*** 

(5.44) 


1,281.14*** 

(5.08) 


Student-level variables 


Male 


-31.07*** 

(4.02) 


25.09*** 

(3.54) 


Eligible for free or reduced-price 
lunch 


-38.90*** 

(5.06) 


-24.70*** 

(4.45) 


Special education 


-218.60*** 

(5.40) 


-189.72*** 

(4.76) 


Current English language learner 


-237.81*** 

(5.65) 


-151.44*** 

(4.98) 


Former English language learner 


-51.02*** 

(6.34) 


-24.50*** 

(5.58) 


Migrant 


-18.40 

(12.93) 


1.14 

(11.38) 


School-level variables 


Percentage of grade 8 students 
receiving special education 
services 


-4.47 

(7.28) 


-5.26 

(6.60) 


Percentage of grade 8 English 
language learner students 


3.83 

(6.95) 


6.17 

(6.37) 



* Significant at p = 0.05; *** significant at p = 0.001. 

Note: Standard errors are in parentheses. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education and 
U.S. Department of Education (2008). 



points lower on the reading test and 
151 points lower on the math test than 
were Hispanic students who had never 
been identified as ELL students. 

• Hispanic students who were former 
ELL students were predicted to score 
51 scaled score points lower on the 
reading test and 25 points lower on the 
math test than were non-ELL students. 

School-level variables 

One school-level variable was associated with 
Hispanic students’ test scores and was statisti- 
cally significant at thep = 0.001 level (see table 5): 



• for every 10 percentage point increase 
in the proportion of students eligible 
for free or reduced-price lunch in a 
school over the mean of all schools, 
Hispanic students were predicted to 
score 17 scaled score points lower in 
reading and 13 points lower in math. 

There were also two statistically significant 
associations at thep = 0.05 level between His- 
panic students’ test scores and school-level vari- 
ables (see table 5): 

• for every 100-student increase in a 
school’s student population over the 
mean of all schools, Hispanic students 
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were predicted to score 2.83 scaled 
score points lower in math. 

• Hispanic students attending schools 
with no dropouts were predicted to 
score 25 scaled score points higher in 
reading than were Hispanic students 
attending schools with dropouts. 

No statistically significant association was 
found between six variables and Hispanic stu- 
dents’ test scores in either reading or math. 
These variables include one student-level vari- 
able (whether the Hispanic student is also clas- 
sified as a migrant child) and five school-level 
variables (percentage of grade 8 ELL student, 
percentage of grade 8 students receiving special 
education services, percentage of Hispanic stu- 
dents, student-teacher ratio, and school locale). 

The student- and school-level variables in 
the model explained about 24 percent of the 
variation in reading test scores and about 17 
percent of the variation in math test scores. 

Conclusions 

This study adds to the research available to 
Pennsylvania policymakers and communities 
as they seek to increase the achievement of the 
state’s Hispanic student population, provid- 
ing them with more information than simply 
English language proficiency. This study sug- 
gests that decisionmakers remain mindful of 
the demographic makeup or racial/ethnic pro- 
portions of schools, whether the schools have 
dropouts, the size of the schools, and students’ 
English language ability and socioeconomic 
status (measured through eligibility for free 
or reduced-price lunch in this study) when 
developing scaffolding strategies and instruc- 
tion proposed to narrow the achievement gap 
between Hispanic students and other students. 

This study used 2007/08 Pennsylvania 
grade 8 student performance data to compare 
the academic achievement of the Hispanic 
student subgroup against the achievement 
of other student subgroups. Hispanic grade 
8 students in Pennsylvania had lower test 



Study limitations 



scores than non-Hispanic students in both 
reading and math. When the non-Hispanic 
subgroup was disaggregated into White stu- 
dents, Black students, and other racial/ethnic 
minority students, Hispanic students were 
found to score nearly the same on the reading 
test as Black students and lower than White 
students and other racial/ethnic minority 
students. Hispanic students outperformed 
Black students in math by 27 scaled score 
points. The authors conclude that the His- 
panic performance gaps identified through 
the 2009 NAEP are supported in the PSSA 
data and that there were observed differences 
in academic performance based on a student’s 
self-reported ethnicity. 

This study also investigated the relation- 
ship between Hispanic student performance 
and several student- and school-level factors. 
While 7 of the 14 factors had statistically sig- 
nificant associations, the two with the great- 
est apparent relationship with student perfor- 
mance were current English language learner 
status and special education status. A His- 
panic student who was also a current ELL stu- 
dent was predicted to score 238 scaled score 
points lower in reading and 151 points lower 
in math than a non-ELL Hispanic student. A 
Hispanic student who was also receiving spe- 
cial education services was predicted to score 
219 scaled score points lower in reading and 
190 points lower in math than a Hispanic stu- 
dent who was not receiving special education 
services. 

Study limitations 

The study methods cannot identify causal 
relationships between variables. The statistics 
represent associations between selected inde- 
pendent variables and the dependent variable 
of Hispanic students’ test scores. The analysis 
provides no information on the cause or reason 
for the test performance of Hispanic students. 
The fact that an association between test per- 
formance and ELL status was identified should 
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not be interpreted to mean that ELL status 
causes lower test performance. 

Any findings related to dropouts should 
be interpreted with caution because of the lack 
of a common approach to counting dropouts. 
This study treated the existence of dropouts in 



Study limitations 



a school as a dichotomous variable and did not 
attempt to explore differences in dropout rates. 

Finally, the findings are based on data from 
the population of grade 8 students in Pennsyl- 
vania in 2007/08 and should not be generalized 
to other states, grades, or school years. 
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Appendix A 

Summary of previous research 
on Hispanic students' academic 
achievement 

Previous research on Hispanic students has 
identified several factors associated with their 
academic achievement, including gender, socio- 
economic status, English language proficiency, 
and migrant status. 

• Gender. Previous studies found higher 
math achievement among Hispanic 
males than among Hispanic females 
(Freeman 2004; McGraw, Lubienski, 
and Strutchens 2006; Pong 2010). This 
difference is especially large for stu- 
dents with high socioeconomic status. 
However, other research suggests no 
significant gender difference in read- 
ing performance among Hispanic high 
school students (LoGerfo, Nichols, 
and Chaplin 2006). 

• Socioeconomic status. Poverty has long 
been recognized as a risk factor for 
education failure (Luthar 1999; Weiss- 
bourd 1996). Hispanic students from 
higher socioeconomic backgrounds(as 
measured by household income; Sirin 
2005; White 1982) tend to have 
greater math proficiency than their 
lower socioeconomic counterparts 
(Pong 2010; Reardon and Galindo 
2007). Previous research suggests His- 
panic students’ low socioeconomic 
status is one reason their school per- 
formance has lagged behind that 
of White students (Pong, Hao, and 
Gardner 2005). Studies that focused 
on Mexican students found that socio- 
economic status is a significant pre- 
dictor of math learning in elementary 
school (Crosnoe 2006) and of math 
and reading performance in middle 
school (Ream 2005). 

• English language proficiency. Hispanic 
students with difficulty in the English 
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language have lower performance in 
reading and math than Hispanic stu- 
dents who are proficient in English 
(Eamon 2005; Reardon and Galindo 
2007; Terwilliger and Magnuson 
2005). 

• Special education status. Sanchez et al. 
(2009) found that Hispanic students 
receiving special education services 
have lower math and English language 
arts (reading) test scores than do other 
Hispanic students. 

• Migrant status. Hispanic students 
whose parents do seasonal migrant 
work face many challenges in educa- 
tion, including frequent interruptions 
in schooling (Suarez-Orozco, Gaytan, 
and Kim 2010). Several studies without 
racial/ethnic breakdowns link schools 
with higher student mobility to lower 
academic achievement (Hanushek, 
Kain, and Rivkin 2004; Ream 2005; 
Rumberger 2002). Although there is 
no definitive count of migrant workers 
and children in Pennsylvania, a recent 
study noted there were 45,000-50,000 
migrant farm workers harvesting fruit, 
mushrooms, and vegetables (Cason 
and Snyder 2004). 

Since the 1960s, researchers have found 
school social context to be important in chil- 
dren’s intellectual development. However, 
few studies on school effects have focused on 
Hispanic students. The salient findings on 
school contexts related to student achievement 
include: 

• Economic status of the school popula- 
tion. Students attending schools with 
a greater percentage of students from 
high-income family backgrounds 
tended to show higher academic 
achievement (Gamoran 1992; Pong 
1998; Willms 1992). By contrast, stu- 
dents attending schools with a greater 
percentage of students eligible for free 
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or reduced-price lunch tended to score 
lower on standardized tests (Sirin 
2005). 

• Percentage of racial/ethnic minority stu- 
dents. Several studies found that stu- 
dents had lower academic performance 
in schools with high concentrations of 
racial/ethnic minority students (Cole- 
man 1966; Hess and Warden 1988; 
Pong 1998; Rumberger and Willms 
1992). Sanchez et al. (2009) found that 
Hispanic students attending a school 
with a high percentage of Hispanic 
students tended to have lower math 
and reading test scores. 

• Percentage of students receiving special 
education services. Kalambouka et al. 
(2007) found that students who did 
not have special education needs were 
unaffected by attending a school with 
a high proportion of special education 
students. Sanchez et al. (2009) found 
no association between the proportion 
of special education students in the 
school and grade 10 Hispanic students’ 
math and reading achievement scores 
in three of four school years . 

• Percentage of English language learner 
students. Hispanic students attend- 
ing schools with high concentrations 
of English language learner students 
(Orfield, Yun, and Project 1999; 
Schmid 2001; Van Hook and Balistreri 
2002) are more likely to have less expe- 
rienced teachers (Rivkin, Hanushek, 
and Kain 2005), which may affect 
their learning. However, schools with 
higher concentrations of English lan- 
guage learner students also have more 
Title 1 services, are more likely to offer 
support and remedial programs and 
native language instruction, are more 
likely to use standardized procedures 
to identify ELL students, and are more 
likely to be involved in parent outreach 
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and support activities than schools 
with lower concentrations (Cohen, 
Deterding, and Clewell 2005). While 
Crosnoe (2005) found no significant 
relationship between Mexican stu- 
dents’ math performance and the per- 
centage of students enrolled in English 
as a second language classes, Sanchez et 
al. (2009) found positive associations 
between Hispanic students’ math and 
reading achievement and the percent- 
age of English language learner stu- 
dents in a school in one of three school 
years. 

• Dropout rates. Hispanic students have 
higher dropout rates than other racial/ 
ethnic groups (Fry 2003; Oropesa and 
Landale 2009). Sanchez et al. (2009) 
found that Hispanic students in 
schools with higher dropout rates had 
lower math and reading performance 
than their Hispanic peers in schools 
with lower dropout rates. 

• School size. The literature on the rela- 
tionship between school size and stu- 
dent achievement is mixed. While 
some studies found no significant rela- 
tionship (Gardner 2001; Milesi and 
Gamoran 2006) or even a positive rela- 
tionship (Wyse, Keesler, and Schnei- 
der 2008), most reported a negative 
relationship (Caldas 1993; Fowler 
and Walberg 1991; Lee and Smith 
1997; McMillen 2004). A recent study 
found that, while Hispanic students 
from Mexican immigrant families 
had lower math scores in larger than 
in smaller schools, other Hispanic stu- 
dents scored higher in larger schools 
(Crosnoe 2005). 

• Student-teacher ratio. The student- 
teacher ratio is an indicator of class 
size per teacher. Earlier correlational 
studies of class-size effect on student 
achievement found that reducing class 
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size did not increase student achieve- 
ment (Porwoll 1978; Hallinan and 
Sorensen 1985). Since the late 1990s, 
Tennessee’s Project STAR, a longitudi- 
nal study of math and reading achieve- 
ment based on a randomized experi- 
ment, has dominated the discussion 
on class size. Project STAR provided 
evidence that small class size increases 
student math performance in the pri- 
mary years (Finn and Achilles 1999). 
Using the STAR data, researchers 
found a statistically significant posi- 
tive effect on racial/ethnic minority 
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students’ reading performance in small 
class sizes (Nye, Hedges, and Konstan- 
topoulos 2004). 

• School locale. Previous research has 
not found a significant association 
between the geographic location of 
the school (urban, suburban, town, or 
rural) and student achievement overall 
(Pong 1998) or achievement of His- 
panic students (Crosnoe 2005; San- 
chez et al. 2009). 

For a more detailed review of the literature, 
see Sanchez et al. (2009) and Suarez-Orozco, 
Gaytan, and Kim (2010). 
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Appendix B. Data removal process 

This study used the following two datasets: 
the Pennsylvania System of School Assessment 
(PSSA) dataset, obtained from the Pennsylva- 
nia Department of Education, and the Com- 
mon Core of Data of the National Center for 
Education Statistics (U.S. Department of Edu- 
cation 2008). The PSSA dataset was used for 
both the descriptive analysis and the multilevel 
modeling; the Core of Data was used solely for 
multilevel modeling. The PSSA dataset con- 
tained information on both Hispanic and non- 
Hispanic students, and the Common Core of 
Data contained information only on Hispanic 
students. The Common Core of Data also 
contained school-level variables matched from 
other sources. This appendix examines how 
missing data were handled. 



they were an English language learner 
(ELL) student in the first year of 
enrollment in a U.S. school. 

• Students who did not take one or 
both of the PSSA tests due to medical 
emergency. 

Applying these criteria lowered the number 
of eligible students to 142,333. Of these, 1,366 
were missing either an English language arts 
(reading) or math score. Excluding students 
with missing test scores, which constituted 
less than 1 percent (0.97) of eligible cases, was 
not expected to affect the descriptive results. 
Removing these students brought the number 
of cases for the descriptive analysis down to 

140,967 (table Bl). 

Data were removed in two steps (table 
B2). Step 1 removed the ineligible cases. 



Missing data 

The PSSA dataset has information on 148,360 
grade 8 students for the 2007/08 school year. 
Some students were removed from the analysis 
for the following reasons: 

• Students attending a school that did not 
meet school inclusion criteria. That is, 
they were attending nonpublic schools^ 
or were in education settings not 
defined as schools, such as special pro- 
grams, centers, or units (including inter- 
mediate units, juvenile correction facili- 
ties, private or nonpublic schools, and 
alternative programs without a Pennsyl- 
vania Department of Education-issued 
administrative unit number). The 
excluded schools were not found in the 
Common Core of Data either. 

• Students who were excluded from 
testing because of parental request, 
because they were assessed under 
the Pennsylvania Alternate System 
of Assessment, because of extended 
absence, because they should have but 
had not yet received an Individual- 
ized Education Program, or because 



TABLE Bl 

Data removal information for descriptive analysis, Pennsylvania 
System of School Assessment 2007/08 


Data element 


Number 


Initial number of grade 8 students 


148,360 


Exclusion by design 


Attended nonpublic or nonschool program 


4,256 


Students not assessed 


Parental request 


48 


Pennsylvania Alternate System of Assessment 


1,054 


Extended absence 


371 


Any combination of the above 


3 


Court/agency placed but did not receive 
individualized education program 


10 


English language learner student in first year of 
enrollment in United States 


144 


Medical emergency 


141 


Total eligible cases 


142,333 


Missing test score data, unknown reason 


Missing English language art test results 


1,184 


Missing math test results 


992 


Missing either score 


1,366 


Final number of cases for analysis 


140,967 


Source; Authors' analysis of data from the 2007/08 Pennsylvania 
ment provided by the Pennsylvania Department of Education. 


System of School Assess- 
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TABLE B2 

Demographic and background characteristics 
before and after each step of data removal, 
Pennsylvania System of School Assessment 
2007/08 (percent) 





Total 


Data 


Data 




grade 8 


removal 


removal 


Characteristic 


students 


step 1 


step 2 


Gender 


Male 


51.7 


51.2 


51.1 


Female 


48.3 


48.9 


48.9 


Free or reduced-price lunch 


Eligible 


33.8 


33.1 


32.9 


Not eligible 


66.2 


66.8 


67.1 


Special education status 


Receiving special education 








services 


17.4 


14.9 


14.7 


Not receiving special 
education services 


82.6 


85.1 


85.3 


English language learner status 


Current English language 
learner 


1.9 


1.8 


1.8 


Former English language 
learner 


1.4 


1.4 


1.4 


Never an English language 
learner 


96.7 


96.8 


96.8 


Migrant status 


Migrant 


0.2 


0.2 


0.2 


Nonmigrant 


99.8 


99.8 


99.8 


Number of observations 


148,360 


142,333 


140,967 



Source: Authors' analysis of data from the 2007/08 Pennsylvania System 
of School Assessment provided by the Pennsylvania Department of 
Education. 



TABLE B3 



Correlations between missing test scores and 
other student-level variables, 2007/08 


Variable 


Missing 

reading 

score 


Missing 

math 

score 


Missing test scores 


Missing math 


0.871 


1.000 


Missing reading 


1.000 


0.871 


Ethnicity 


Black 


0.031 


0.014 


Flispanic 


0.042 


0.043 


White 


-0.060 


-0.046 


Other 


0.018 


0.006 


Gender 


Male 


0.025 


0.025 


Special education status 


Receiving special education 
services 


0.152 


0.166 


Free or reduced-price lunch program eligibility 


Eligible 


0.066 


0.062 


English language learner status 


Current English language learner 
student 


0.061 


-0.001 


Former English language learner 
student 


-0.001 


0.001 


Never an English language learner 
student 


-0.046 


0.000 


Migrant status 


Migrant 


0.013 


-0.001 



Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of 
School Assessment provided by the Pennsylvania Department of Education, 



including students in nonregular schools, stu- 
dents excluded from testing assessment, and 
students who did not take the test due to medi- 
cal emergency. Step 2 removed students who 
had missing test scores. 

The correlations between each missing test 
score and student-level variables are shown in 
table B3. The strongest correlation was between 
the two test scores, suggesting that students 
who missed one test were highly likely to miss 
the other. The correlations between special 



education status and the missing test scores 
were 0.15 for reading and 0.17 for math. Both 
correlations were weak. All other correlations 
were even weaker, at less than 0.07 in absolute 
value. 

Multilevel modeling 

The descriptive analysis used data for both 
Hispanic and non-Hispanic students, but the 
multilevel modeling analyzed Hispanic stu- 
dents only. The dataset used for multilevel 
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modeling had 9,040 students (a subset of the 
full dataset of 140,967). These data came from 
both the PSSA dataset and the Common Core 
of Data. The PSSA data provided student-level 
variables. School-level variables were either 
constructed from aggregating information 
from the PSSA or extracted from the Com- 
mon Core of Data. 

Gender was the only student-level variable 
with missing data. A dummy variable indicat- 
ing missing gender was included in the mul- 
tilevel regression to account for “missing,” so 
that no student-level data were removed. Two 
school-level variables were constructed from 



the PSSA: the percentage of grade 8 students 
receiving special education services, and the 
percentage of grade 8 English language learner 
students. There were no missing data. 

However, there were many missing values 
for the school-level variables from the Com- 
mon Core of Data. Multiple imputation was 
used to impute values for these variables. The 
imputation procedure created five school-level 
datasets — each merged with the student-level 
dataset — creating five datasets that had both 
student- and school-level variables. Appendix C 
provides more information on the imputation 
procedure. 
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Appendix C 
Multiple imputation 

This appendix describes how multiple imputa- 
tion (Rubin 1987; Schafer 1997) was applied to 
address the missing school-level data in the Com- 
mon Core of Data. Whereas there was not a sub- 
stantial amount of data missing from the student- 
level Pennsylvania System of School Assessment 
(PSSA) dataset, values were missing in 33 percent 
of the 604 schools in the data from the Common 
Core of Data. Missing data could lead to biased 
estimates of the association between school vari- 
ables and student test scores. This problem can be 
remedied by imputing the missing data. 

Multiple imputation is a simulation approach 
to analyzing incomplete data. It assumes the 
mechanism of missingness to be “missing at ran- 
dom.” This means that there is a systematic pat- 
tern of missingness (for example, missing on 
income correlated with low level of education) 
and that missing data can be correlated with other 
variables in the dataset. Missing at random is the 
most common imputation assumption for multi- 
variate analysis (Allison 2002; Graham 2009). 

Multiple imputation replaces each missing 
observation with a number of simulated val- 
ues, creating multiple datasets. This method 
creates multiple imputations by using simula- 
tions from a Bayesian prediction distribution 
for normal data and includes two steps: draw- 
ing values for the variables with missing values 
for a particular observation from a conditional 
distribution given the variables with observed 
values; and simulating the posterior population 
means, variances, and covariances from the 
complete sample estimates. The first step is then 
repeated using these new estimates. These two 
steps are repeated until the iterates converge to 
their stationary distribution — to reliably create 
a multiply imputed dataset (Schafer 1997). 

The multiple imputation method is supe- 
rior to single imputation methods, such as 
mean substitution or regression imputation 
(Allison 2002; Graham 2009). Single imputa- 
tion methods tend to produce biased estimates. 
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and the standard errors are biased downward. 
The multiple imputation method repeats the 
imputation process several times so that the 
point estimates are unbiased and the standard 
errors across imputations can be used to adjust 
the standard errors upward (Allison 2002). 

Researchers have suggested that using aux- 
iliary variables, including interaction terms, 
may reduce bias and increase efficiency (Collins 
et al. 2001). In the imputation model, covari- 
ates that were available but not in the multi- 
level models were used to help predict missing 
values. There were 1 1 auxiliary variables. Five of 
the 1 1 were constructed from averaging student 
data from the PSSA. 

• Percentage of grade 8 students who were 
eligible for free or reduced-price lunch. 

• Percentage of grade 8 students who 
were Hispanic students. 

• Percentage of grade 8 students who 
were Black students. 

• Percentage of grade 8 students who 
were White students. 

• Percentage of grade 8 students who 
were migrant students. 

The other six auxiliary variables were 
extracted from the Common Core of Data: 

• Lowest grade in the school. 

• Highest grade in the school. 

• Title 1 school (all students designated 
as eligible for participation in pro- 
grams authorized by Title I of Public 
Law 103-382). 

• Magnet school. 

• Charter school. 

• School dropout rate in 2006/07. 

Additionally, interactions of available stu- 
dent- and school-level covariates were used in 
the imputation procedure. There were 20 inter- 
action terms in the imputation model, each 
constructed by multiplying two variables. One 
variable was at the school-level (from the Com- 
mon Core of Data) and the other represented 
the school- average of student-level data (from 
the PSSA). For example, the percentage of grade 
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8 ELL students and school size were multiplied 
to create an interaction variable. Lour variables 
from averaging the PSSA data were used: per- 
centage of grade 8 ELL students, percentage 
of grade 8 students receiving special educa- 
tion services, percentage of grade 8 students 
from low-income households, and percentage 
of grade 8 Hispanic students. All but one of 
these student-level variables were used in the 
multilevel models to predict test scores for His- 
panic students. Migrant status was not used to 
construct interaction variables because the per- 
centage of migrants in the student population 
was small (about 3 percent). Live of the eight 
school-level variables (from the Common Core 



of Data) were used in the multilevel models: 
number of students receiving free school lunch, 
school size, total number of Hispanic students, 
student-teacher ratio, and school dropout rate 
for 2007/08. The other three variables were not 
used because they did not contain missing data 
and were entered in the imputation model as 
main effects only. With four variables from the 
PSSA and five from the Common Core of Data, 
there were a total of 20 interaction variables. 

Before multiple imputation, the amount of 
missing data for the original variables (excluding 
interaction terms) was examined (table Cl). Ten 
school-level variables had no missing values: school 
locale, magnet school, charter school, and seven 



TABLE Cl 

Missing data for each variable in the school dataset, 2007/08 


School variable 


Number of 
nonmissing 


Number of 
missing 


Percent missing 


Main variables 


Percentage of grade 8 English language learner students 


604 


0 


0.00 


Percentage of grade 8 students receiving special education services 


604 


0 


0.00 


Number of Hispanic students 


586 


18 


2.98 


Number of students eligible for free or reduced-price lunch 


510 


94 


15.56 


Student-teacher ratio 


591 


13 


2.15 


School size 


596 


8 


1.32 


Dropout rate 


594 


10 


1.66 


School locale 


604 


0 


0.00 


Auxiliary variables 


Percentage of grade 8 students eligible for free or reduced-price lunch 


604 


0 


0.00 


Percentage of grade 8 Hispanic students 


604 


0 


0.00 


Percentage of grade 8 Black students 


604 


0 


0.00 


Percentage of grade 8 White students 


604 


0 


0.00 


Percentage of grade 8 migrant students 


604 


0 


0.00 


Percentage of grade 8 English language learner students 


604 


0 


0.00 


Lowest grade in school 


599 


5 


0.83 


Highest grade in school 


599 


5 


0.83 


Title 1 school 


488 


116 


19.21 


Magnet school 


604 


0 


0.00 


Charter school 


604 


0 


0.00 


Dropout rate 2006/07 


603 


1 


0.17 


Source; Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsyl 
U.S. Department of Education (2008). 


Ivania Department of Education and 
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variables representing the percentages of grade 8 
student characteristics (English language learner 
students, students receiving special education ser- 
vices, students eligible for free or reduced-price 
lunch, and Black, Hispanic, White, and migrant 
students). The variable Title I school had the 
largest number of missing values — 19.2 percent 
of the 604 schools did not report Title I status. 

Overall, 404 schools had no missing val- 
ues, 95 schools had 1-2 missing values, and 
another 95 schools had 5-10 missing values. A 
small number of schools (10 schools) had more 
than 10 missing values; eight of those schools 
had predominately missing values (28 or 30). 
These 8 schools were excluded for the sensitiv- 
ity analysis (appendix D). 

The imputation model was estimated using 
the command “ICE” (imputation by chain equa- 
tions) in STATA (Royston 2004). Depending on 
the variables to be imputed, three types of mod- 
els were used: ordinary least squares regression 
for continuous variables, such as the percent- 
age of students eligible for free or reduced-price 
lunch or the dropout rate;^ logistic regression 
for dichotomous variables, such as charter or 
magnet school; and multinomial logistic regres- 
sion for categorical variables, such as the school s 
location in urban, suburban, town, or rural 
areas. All imputed variables were originals from 
the PSSA or Common Core of Data. 



The imputation model produced five 
imputed datasets. Each dataset was analyzed 
in an identical fashion and was then combined 
to yield a single set of results based on Rubin’s 
rules (Rubin 1987). The point estimate for a 
particular parameter was the average of the 
five estimates from the five imputed datasets. 
The variance estimate was calculated from the 
within-imputation and between-imputation 
variances, both weighted by the number of 
imputations (Schafer 1997). 

One way to examine whether the multiple 
imputations were done properly is to inspect 
the mean and standard deviation of each origi- 
nal and imputed variable to see whether the 
statistics are similar. The school-level variables 
that were used in the multilevel model analy- 
sis and that had missing values were examined 
(table C2). The average values of each school 
variable before and after imputation were very 
similar. The standard deviations were also simi- 
lar, except for the percentage of students eligi- 
ble for free or reduced-price lunch, which had a 
larger standard deviation after imputation. This 
result is not surprising given that the amount 
of missingness was highest for the percentage of 
students eligible for free or reduced-price lunch 
(about 16 percent). Overall, the distributions 
of the school variables with missing data were 
similar before and after multiple imputation. 



TABLE C2 

Comparison between school means, with and without imputation, school-level data 2007/08 





With imputation 


Without imputation 




School characteristics (n = 604) 


Mean 


Standard 

deviation 


Mean 


Standard 

deviation 


Percent 

missing 


Percentage of Hispanic students 


8.97 


17.75 


9.19 


16.63 


2.98 


Percentage of students eligible for free 
or reduced-price lunch 


42.34 


33.18 


40.74 


28.10 


15.56 


Student-teacher ratio 


14.05 


3.76 


14.05 


3.73 


2.15 


School size (number of students) 


683.91 


449.53 


684.07 


447.75 


1.32 


Percentage of school with no dropout 


83.31 


0.38 


84.18 


0.37 


1.66 



Source: Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education and 
U.S. Department of Education (2008). 
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Appendix D 
Sensitivity analyses 

Two sensitivity analyses were conducted. 
The first was restricted to the sample of stu- 
dents with no missing data. Listwise deletion 
was used to construct this sample. A total of 
1,010 students and 105 schools were dropped, 
resulting in a sample of 8,030 students in 499 
schools. Table D1 shows the multilevel regres- 
sion results. 

The coefficients and significance levels of 
the student-level variables were similar to those 
in the main model (table 5 in the main report). 
However, some estimates differed at the school 
level. The percentage of grade 8 students receiv- 
ing special education services had the opposite 
sign in the models predicting both test scores. 
The rural school and school size variables had 
the opposite sign in the models predicting 
math scores. Of these, only school size was 
statistically significant {p = 0.05) in the main 
model. 

Although the percentage of students eligi- 
ble for free or reduced-price lunch was statisti- 
cally significant at thep = 0.001 level in both 
tables 5 and Dl, school locale (suburban school 
for both reading and math, town school for 
reading only, and rural school for math only) 
was significantly associated with test scores 
only in table Dl. Also, school size (math) and 
school without dropouts (reading) were signifi- 
cant in only table 5. 

The results from the first sensitivity analy- 
sis indicate that the findings are sensitive to 
excluding cases based on missing data. 

The second sensitivity analysis was 
restricted to a sample of 8,938 students within 
596 schools. This analysis excluded 102 stu- 
dents in the 8 schools that had mostly miss- 
ing values. The multilevel regression results 
were very similar to the results in table 5, 



TABLE Dl 

Sensitivity analysis based on the sample without missing values 



Statistic and variable 


Reading 


Math 


Standard deviation of test score 


241.679 


199.757 


Intercept 


1289.380*** 


1285.563*** 


Student-level variables 


Male 


-29.959*** 


25.234*** 


Receiving special education services 


-216.566*** 


-189.171*** 


Eligibility for free or reduced-price lunch 


-39.738*** 


-22.927*** 


Current English language learner student 


-240.126*** 


-153.859*** 


Former English language learner student 


-48.487*** 


-23.844*** 


Migrant student 


-16.610 


1.972 


School-level variables 


Percentage of grade 8 students receiving 
special education services 


0.738 


1.051 


Percentage of grade 8 English language 
learner students 


0.397 


2.711 


Percentage of Hispanic students 


-1.143 


-0.406 


Percentage of students eligible for free or 
reduced-price lunch 


-21.962*** 


-14.955*** 


Student-teacher ratio 


0.687 


0.194 


School size 


-0.834 


0.287 


School with no dropouts 


15.179 


25.576* 


Suburban school 


-39.703* 


-23.356 


Town school 


-45.367* 


-35.517* 


Rural school 


-27.284 


-33.902* 


Between-school variance 


100.852*** 


73.955*** 


Within-school variance 


221.780 


187.341 


Total variance 


322.632 


261.296 


Percentage of variances explained by the 
model 


24.686 


16.815 



* Significant at p = 0.05; *** significant at p = 0.001. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assess- 
ment provided by the Pennsylvania Department of Education, and U.S. Department of 
Education (2008). 



suggesting that dropping the 8 schools made 
little difference to the estimation of the associa- 
tion between test scores and the student- and 
school-level characteristics (table D2). 
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TABLE D2 

Sensitivity analysis based on the sample without the eight 
schools that had mostly missing values 



Statistic and variable 


Reading 


Math 


Standard deviation of test score 


241.679 


199.757 


Intercept 


1398.783*** 


1323.521*** 


Student-level variables 


Male 


-30.750*** 


25.307*** 


Receiving special education services 


-218.807*** 


-190.030*** 


Eligibility for free or reduced-price lunch 


-38.781*** 


-24.180*** 


Current English language learner student 


-237.880*** 


-151.263*** 


Former English language learner student 


-50.469*** 


-24.673*** 


Migrant student 


-18.920 


0.849 


School-level variables 


Percentage of grade 8 students receiving 
special education services 


-2.790 


-4.341 


Percentage of grade 8 English language 
learner students 


3.508 


6.221 


Percentage of Hispanic students 


-1.552 


-0.765 


Percentage of students eligible for free or 
reduced-price lunch 


-18.507*** 


-13.587*** 


Student-teacher ratio 


1.039 


0.474 


School size 


-2.622* 


-2.960*** 


School with no dropouts 


20.885 


30.758*** 


Suburban school 


-10.742 


-8.227 


Town school 


-18.004 


-21.931 


Rural school 


5.756 


-8.938 


Between-school variance 


88.178*** 


68.935*** 


Within-school variance 


221.816 


186.432 


Total variance 


309.994 


255.367 


Percentage of variances explained by the 
model 


22.507 


14.950 



* Significant at p = 0.05; *** significant at p = 0.001. 

Source; Authors' analysis of data from the 2007/08 Pennsylvania System of School Assess- 
ment provided by the Pennsylvania Department of Education, and U.S. Department of 
Education (2008). 
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Appendix E 
Study methods 

This study used descriptive and multilevel 
modeling to answer two research questions: 

• How does the performance of Penn- 
sylvania grade 8 Hispanic students on 
the 2007/08 Pennsylvania System of 
School Assessment English language 
arts and math tests compare with that 
of grade 8 non-Hispanic White, Black, 
and other non-Hispanic students? 

• Among Pennsylvania grade 8 Hispanic 
students, which student- and school- 
level characteristics are associated with 
performance on the 2007/08 Penn- 
sylvania System of School Assessment 
English language arts and math tests? 

Datasets for the analyses 

This report uses student- and school-level data 
from three sources (table El). 

Student-level data came from the PSSA 
dataset. In addition to student performance on 
English language arts (reading) and math, the 
PSSA dataset provided information on student 
demographic characteristics (ethnicity, gender. 



special education status, eligibility for free or 
reduced-price lunch, English language learner 
status, and migrant status). Ethnicity was self- 
reported, with students asked to select the eth- 
nicity with which they identify most closely. 
Students choosing the category “two or more 
races” were included in the “other ethnicities” 
subgroup. 

School-level data for 2007/08 for all pub- 
lic schools in Pennsylvania were collected for 
grade 8 students included in the PSSA data- 
base. Student-level data were aggregated or 
averaged within schools to create the school- 
level variables (see below). 

Demographic data for all Pennsylvania pub- 
lic schools were taken from the National Cen- 
ter for Education Statistics 2007/08 Common 
Core of Data (U.S. Department of Education 
2008). The variables, which were comparable 
across schools, included the percentage of His- 
panic students, percentage of students receiving 
free or reduced-price lunch, percentage of stu- 
dents receiving special education services, per- 
centage of English language learner students, 
student-teacher ratio, school size, and school 
locale (urban, suburban, town, rural). 



TABLE El 

Student- and school-level variables included in the multilevel regression model 



Pennsylvania System of 
School Assessment 


Common Core of Data 


Pennsylvania Department 
of Education website 


• Gender 


• Percentage of Flispanic students 


• Dropout 


• Special education status 


• Percentage of students receiving free 




• Eligibility for free or reduced-price 


or reduced-price lunch^ 




lunch^ 


• Student-teacher ratio 




• Current English language learner 


• School size 




student 


• School locale (urban, suburban, town. 




• Former English language learner 
student 

• Migrant student 

• Percentage of students receiving 
special education services 

• Percentage of English language 
learner students 


rural) 





a. Eligibility for free or reduced-price lunch is used as a proxy measure of low-income status. 

Source: Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education and 
U.S. Department of Education (2008). 
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Dropout rates for each school in 2007/08 
were collected from the Pennsylvania Depart- 
ment of Education website. 

The Common Core of Data contains 
unique state school identification numbers, and 
the Pennsylvania Department of Education 
data contains school names, which served as 
linking variables. Both state school identifica- 
tion numbers and school names were available 
in the PSSA data, enabling dataset merging. 

Outcome variables used for the analyses 

Scaled scores for the reading and math tests 
were used in both the descriptive and multi- 
level regression analyses. The scaled score range 
for the reading test was 700-2,646, with a 
mean of 1,443.76 and standard deviation of 
247.17. Half the items on the test were related 
to comprehension and reading skills and the 
remaining items were based on interpretation 
and analysis of fictional and nonfictional text. 

The scaled score range for the math test was 
700-2,259, with a mean of 1,396.08 and stan- 
dard deviation of 220.66. Of the 66 items on 
this test, 14 were related to numbers and opera- 
tions, 10 to measurement, 12 to geometry, 20 
to algebraic concepts, and 10 to data analysis 
and probability. 

More information on the 2007/08 PSSA 
instruments and analyses can be found in the 
Technical Manual for the Pennsylvania System 
of School Assessment (Data Recognition Corpo- 
ration 2009). 

Independent variables used for the analyses 

All six student background variables came 
from the 2007/08 PSSA. 

• Hispanic student. Whether a student 
had self-identified as Hispanic. The 
comparison group is non-Hispanic as a 
single group or White, Black, or “other” 
students (includes several subgroups; for 
example, Asian and Native American). 

• Gender. Whether a student is reported 
as male. The comparison group is 
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female. A small number of students 
did not report their gender. A dummy 
variable was constructed to represent 
missing gender. 

• Special education services. Whether a 
student had an Individualized Edu- 
cation Program (not including gifted 
programs). 

• Eligibility for free or reduced-price 
lunch. Children from families with 
incomes at or below 130 percent of the 
poverty level are eligible for free meals. 
Children from families receiving Tem- 
porary Assistance for Needy Families 
or food stamp benefits automatically 
qualify for free meals. Children from 
families whose income is 130-185 
percent of the poverty level are eligible 
for reduced-price meals (Pennsylvania 
Department of Education n.d. b). 

• English proficiency. Whether a student 
had been assessed and identified as an 
English language learner student in 
2007/08 or as a former English lan- 
guage learner who exited an English 
as a second language/bilingual pro- 
gram before 2007/08. The compari- 
son group is students who had never 
been identified as an English language 
learner student. 

• Migrant status. Refers to “any child 
domiciled temporarily in any school 
district for the purpose of seasonal 
or temporary employment, but not 
acquiring residence therein, and any 
child accompanying his parent or 
guardian who is so domiciled” (Penn- 
sylvania Department of Education 
2008, p. 6). School personnel make 
this designation. 

School-level variables used in the multilevel 
modeling regressions include the following: 

• Percentage of grade 8 students receiving 
special education services. The number 
of grade 8 students receiving special 
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education services in a school divided 
by the total number of grade 8 stu- 
dents and multiplied by 100. 

• Percentage of grade 8 English language 
learner students. The number of grade 
8 students identified by the school as 
current or former English language 
learner students divided by the total 
number of grade 8 students and mul- 
tiplied by 100. 

• Percentage of Hispanic students. The 
number of Hispanic students divided 
by the total number of students in the 
school and multiplied by 100. 

• Percentage of students eligible for free or 
reduced-price lunch. The number of stu- 
dents eligible for free or reduced-price 
lunch divided by the total number of 
students in the school and multiplied 
by 100. 

• Student- teacher ratio. The total num- 
ber of students in the school divided by 
the total number of full-time equiva- 
lent classroom teachers in the school. 

• School size. The total number of stu- 
dents in the school. 

• School without dropouts in 2007/08. 
Dropout rates are often considered a 
measure of school quality. Research has 
shown that dropout rates are negatively 
associated with student achievement 
(see appendix A). Thus, dropout rates 
are a meaningful concept in the study 
of student achievement. Dropouts, 
though more common in high school, 
were also found in middle school. Even 
a small number of dropouts within 
a school can indicate a risk factor for 
students. All schools in Pennsylvania 
reported a dropout rate. The major- 
ity (84 percent) of the schools in the 
dataset had no dropouts (dropout 
rate = 0). Using a linear specification of 
the dropout rate did not reveal any sig- 
nificant relationship between dropout 
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rate and test scores. The dropout rates 
were recoded as a dichotomous variable 
(school without dropouts) to differen- 
tiate schools that had some dropouts 
from schools that had none. 

• School locale. The National Center for 
Education Statistics defines the urban- 
centric locale of schools in 12 catego- 
ries. These categories were collapsed 
into four: urban (large, midsize, and 
small cities), suburban (large, mid- 
size, and small suburbs), town (fringe, 
distant, and remote towns), and rural 
(fringe, distant, and remote rural areas) 
locales. Urban schools are the reference 
group in the multivariate analysis. 

Descriptive analyses 

Summary statistics were used to describe the 
PSSA performance of Hispanic and non- 
Hispanic students, and the descriptive statistics 
provided a demographic profile of Hispanic 
students and non-Hispanic students, either as a 
whole or separately as non-Hispanic White, non- 
Hispanic Black, or other non-Hispanic students. 

The descriptive statistics for all categori- 
cal variables are shown as percentages. These 
include all student-level variables except test 
scores. For continuous variables, which include 
all school-level variables, descriptive statistics 
are shown as means and standard deviations. 

To answer the first research question, the 
differences in the mean scores for Hispanic and 
non-Hispanic students were examined. Sig- 
nificance tests were performed based on ^-tests 
with adjustment for clustering within schools. 

Multilevel modeling procedures 

To answer the second research question, mul- 
tilevel regression modeling was used to exam- 
ine how student and school characteristics 
can account for the performance of Hispanic 
students. The variables used are defined in 
table E2. Because students are clustered within 
schools and share many of the same school 
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experiences of interest to researchers, their 
characteristics are not independent. Cluster 
sampling violates the ordinary least squares 
assumption of independent errors and would 
bias the ordinary least squares estimates. An 
appropriate method for this type of data struc- 
ture is multilevel modeling, which corrects for 
the bias by incorporating a unique random 
effect for each school. Furthermore, the multi- 
level model allows for simultaneous estimation 
of student- and school-level effects by positing 
a set of relationships at both the individual 
student and aggregate levels between schools 
(Raudenbush and Bryk 2002). A two-level 
multilevel model was applied. The first level 
units were students, who were nested within 
the second level units, schools. Formally, a 
student-level equation was specified as: 

^ij = ^oj + + '^2ppecial_ed)j 

+ '^2,j{low_inc)j + ^2^fELL)j 
-I- ^^j{ELL_former) j + ^ ^j^migrani) ■ -I- s- 

where JC were math or reading test scores, 
and £ ■ were student-specific random errors. 
All independent variables in the student-level 
equation were categorical variables, entered 
uncentered in the model. The second level units 
were schools, and the school-level equations 
were specified as: 

= Too + lQi{%special_ed)j + y^^{%ELL)j 
+ 'i Q'^i.dropout) ■ -V yf^^{%Hispanics)j 
yQ^{%low_inc)j + yQid^stu_teach_ratio)j 
+ 7o7(.^t^_«2:d’) . -I- y Q^{locale_urban) ■ 

+ yQC){locale_town)j + y^Q{locale_rura[)j + u^- 

^ 1 - 1-6 j ~ Ti ->-6 0 
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The aim was to examine what student and 
school characteristics were associated with Ffis- 
panic student achievement. The school mean 
achievement (the intercept was specified as 
random and all other first-level slopes as fixed. 
All variables were entered grand-mean centered 
so that the intercept, Yqq, can be interpreted as 
the overall average test score in the data. 

A null multilevel model was first esti- 
mated without any covariates to examine 
the variances within and between schools. 
Model 1 then added all student-level vari- 
ables. Model 2 added school-level variables 
to measure the extent to which student and 
school characteristics are associated with the 
test scores of FFispanic students. The results 
of this full model are presented in table 5 in 
the main report. Two-tailed ^-tests were used 
to show whether the association between the 
test score and a variable was significantly dif- 
ferent from zero at the standard level of sig- 
nificance of 0.05. The percentage of variance 
explained by the model was obtained by com- 
paring the total variances of the null model 
and the variances of the full model. Tables 
E3 and E4 provide more detailed results. The 
null model shows that school-level variables 
explain about 32 percent of reading test scores 
(102.604/323.976 x 100) and 29 percent of 
math test scores (77.849/264.097 x 100). The 
student-level variables explain about 19 per- 
cent of the reading test scores and 14 percent 
of the math test scores. Adding the school-level 
variables improved the explained variances by 
only 4 percent for both test scores. Also, add- 
ing the school-level variables in the model 
did not change the level of significance of the 
student-level variables. 
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TABLE E2 

Independent variables used in multilevel modeling 



Variable Definition 



Student-level variables 




Hispanic student 


1 = Hispanic student; 0 = non-Hispanic student 


Non-Hispanic White student 


1 = non-Hispanic White student; 0 = not non-Hispanic White student 


Non-Hispanic Black student 


1 = non-Hispanic Black student; 0 = not non-Hispanic Black student 


Other non-Hispanic student^ 


1 = other non-Hispanic student; 0 = not other non-Hispanic student 


Gender 


1 = male student; 0 = female student 


Special education status^ 


1 = receiving individualized education program ; 0 = not receiving individualized 
education program 


Eligible for free or reduced-price lunch 


1 = Eligible student; 0 = not eligible student 


Not English language learner student 


1 = Not English language learner student; 0 = English language learner student 
(reference) 


Current English language learner student 


1 = current English language learner student who has been assessed and identified 
as English language learner; 0 = not current English language learner student 


Former English language learner student 


1 = former English language learner student who has exited a English as a second 
language/bilingual program; 0 = not former English language learner student 


Migrant status 
School-level variables 


1 = migrant student; 0 = not migrant student 


School size 


Continuous variable; centered around the average school size across all schools in 
the data (unit of change = 100 students) 


Percentage of grade 8 English language 
learner students 


Continuous variable; centered around the average percentage of grade 8 English 
language learner students across all schools in the data (unit of change = 10 
percentage points) 


Percentage of grade 8 students receiving 
special education services 


Continuous variable; centered around the average percentage of grade 8 
students receiving special education services across all schools in the data (unit 
of change = 10 percentage points) 


Percentage of Hispanic students 


Continuous variable; centered around the average percentage of Hispanic 
students across all school in the data (unit of change = 10 percentage points) 


Percentage of students eligible for free or 
reduced-price lunch 


Continuous variable; centered around the average percentage of free or reduced- 
price lunch eligible students across all schools in the data (unit of change = 10 
percentage points) 


Student-teacher ratio 


Continuous variable; centered around the average student-teacher ratio across 
all schools in the data (unit of change = 1 student per teacher) 


School without dropouts 


1 = school dropout rate is zero; 0 = school dropout rate is not zero 


Urban school 


1 = urban (large, midsize, and small cities); 0 = not urban (reference) 


Suburban school 


1 = suburban (large, midsize, and small suburbs); 0 = not suburban 


Town school 


1 = town (fringe, distant, and remote towns); 0 = not town 


Rural school 


1 = rural (fringe, distant, and remote rural areas); 0 = not rural 



a. Students self-report as one of six ethnic groups: American Indian/Alaskan Native, Asian/Pacific Islander, non-Hispanic Black/African American, Latino/His- 
panic, non-Hispanic White, and multiracial. In this report, "other non-Hispanic" includes American Indian/Alaskan Native, Asian/Pacific Islander, and multiracial 
groups. 

b. Excludes gifted programs. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education. 
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Appendix E 



TABLE E3 

Multilevel regression results of the association between student- and school-level characteristics and grade 8 
Hispanic students' scores on the Pennsylvania System of School Assessment reading tests, 2007/08 (n = 9,040) 







Null model 






Model 1 






Model 2 




Statistic and variable 


Coefficient 


Standard 

error 


Signifi- 
cance level 


Coefficient 


Standard 

error 


Signifi- 
cance level 


Coefficient 


Standard 

error 


Signifi- 
cance level 


Intercept 


1,332.078 


6.035 


0.000 


1,317.290 


4.806 


0.000 


1,286.560 


5.458 


0.000 


Student-level variables 


Gender 








-30.656 


4.032 


0.000 


-31.071 


4.020 


0.000 


Missing gender 








-205.737 


202.485 


0.310 


-259.342 


197.163 


0.189 


Special education 








-217.983 


5.415 


0.000 


-218.599 


5.407 


0.000 


Eligibility for free or 
reduced-price lunch 








-47.564 


5.058 


0.000 


-38.900 


5.068 


0.000 


Current English language 
learner student 








-239.289 


5.658 


0.000 


-237.857 


5.658 


0.000 


Former English language 
learner student 








-52.142 


6.359 


0.000 


-51.020 


6.344 


0.000 


Migrant 








-15.308 


12.965 


0.238 


-18.396 


12.934 


0.155 


School-level variables 


Percentage of grade 8 
special education students 
















-4.467 


7.299 


Percentage of grade 8 
English language learner 
students 
















3.828 


6.953 


Percentage of Hispanic 
students 














-1.628 


3.200 


0.611 


Percentage of students 
eligible for free or reduced- 
price lunch 














-17.065 


2.428 


0.000 


Student-teacher ratio 














0.774 


1.455 


0.595 


School size 














24.743 


12.520 


0.049 


School with no dropouts 














-2.294 


1.247 


0.066 


Suburban school 














-1.572 


14.224 


0.912 


School in town 














-10.279 


18.021 


0.569 


Rural school 














13.550 


17.116 


0.429 


Variance components 


Between-school variance 


102.604*** 




77.522*** 




58.127*** 




Within-school variance 


221.372 






186.961 






186.912 






Total variance 


323.976 






264.483 






245.039 






Total variance explained 
(percent) 


na 






18.363 






24.365 







na is not applicable. 

*** Significant at p = 0.01 . 

Source; Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education and 
U.S. Department of Education (2008). 
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Appendix E 



TABLE E4 

Multilevel regression results of the association between student- and school-level characteristics and grade 8 
Hispanic students' scores on the Pennsylvania System of School Assessment math tests, 2007/08 (n = 9,040) 







Null model 






Model 1 






Model 2 




Statistic and variable 


Coefficient 


Standard 

error 


Signifi- 
cance level 


Coefficient 


Standard 

error 


Signifi- 
cance level 


Coefficient 


Standard 

error 


Signifi- 
cance level 


Intercept 


1306.810 


4.748 


0.000 


1298.950 


4.099 


0.000 


1281.140 


5.084 


0.000 


Student-level variables 


Gender 








25.229 


3.546 


0.000 


25.088 


3.539 


0.000 


Missing gender 








-217.024 


177.122 


0.221 


-264.080 


174.300 


0.130 


Special education 








-189.194 


4.762 


0.000 


-189.717 


4.762 


0.000 


Eligibility for free or 
reduced-price lunch 








-30.511 


4.417 


0.000 


-24.700 


4.458 


0.000 


Current English language 
learner student 








-151.954 


4.973 


0.000 


-151.443 


4.983 


0.000 


Former English language 
learner student 








-25.006 


5.590 


0.000 


-24.505 


5.587 


0.000 


Migrant 








2.480 


11.401 


0.828 


1.135 


11.386 


0.944 


School-level variables 


Percentage of grade 8 
special education students 
















-5.255 


6.586 


Percentage of grade 8 
English language learner 
students 
















6.172 


6.377 


Percentage of Hispanic 
students 














-0.705 


2.935 


0.810 


Percentage of students 
eligible for free or reduced- 
price lunch 














-12.602 


2.212 


0.000 


Student-teacher ratio 














0.372 


1.305 


0.776 


School size 














33.296 


11.192 


0.003 


School with no dropouts 














-2.825 


1.109 


0.011 


Suburban school 














-2.552 


12.806 


0.842 


School in town 














-16.437 


16.119 


0.308 


Rural school 














-3.088 


15.633 


0.843 


Variance components 


Between-school variance 


77.849*** 




65.355*** 




54.831*** 




Within-school variance 


186.248 






164.542 






164.462 






Total variance 


264.097 






229.897 






219.293 






Total variance explained 
(percent) 


na 






12.950 






16.965 







na is not applicable. 

*** Significant at p = 0.01. 

Source.- Authors' analysis of data from the 2007/08 Pennsylvania System of School Assessment provided by the Pennsylvania Department of Education and 
U.S. Department of Education (2008). 
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Notes 



Notes 

1. This study uses eligibility for free or 
reduced-price lunch is used as a proxy 
for socioeconomic status. 

2. Eighty-four percent of the schools in 
this study reported no dropouts, and 
an initial analysis found no statisti- 
cally significant relationship between 
a school’s reported dropout rate and 
Hispanic student performance. The 
relationship was revealed when the 
variable was recoded as a dichotomous 
variable indicating only whether or not 
dropouts were reported for a particular 
school. See appendix E for a discussion 
of the variables used in this study. 

3. NAEP scaled scores in math and read- 
ing typically range from 0 to 500. 

4. In addition, every Pennsylvania stu- 
dent in grade 5, 8, or 11 is assessed in 
writing; every student in grade 4, 8, or 
11 is assessed in science 

5. Ethnic groups other than Hispanic, 
Black, and White had small numbers 
in the dataset and were aggregated into 
“other ethnicities” to reduce the risk of 
disclosure. Student ethnicity is deter- 
mined in Pennsylvania through stu- 
dent self-reports. A student indicating 
more than one ethnicity was put into 
the “other ethnicities” subgroup. 

6. Clewell 2005; Cohen, Deterding, and 
Fry 2003; Coleman 1966; Crosnoe 



2005; Finn and Achilles 1999; Gamo- 
ran 1992; Hallinan and Sorensen 
1985; Hess and Warden 1988; Oropesa 
and Landale 2009; Pong 1998; Por- 
woll 1978; Rumberger and Sirin 2005; 
Suarez-Orozco, Gaytan, and Kim 
2010; Rivkin, Hanushek, and Kain 
2005; Milesi and Gamoran 2006; Nye, 
Hedges, and Konstantopoulos 2004; 
Wyse, Keesler, and Schneider 2008; 
Willms 1992. 

7. Eighty-four percent of the schools 
included in this study reported no 
dropouts, and an initial analysis 
found no statistically significant rela- 
tionship between a school’s reported 
dropout rate and Hispanic student 
performance. Since previous research 
indicated that school-level dropout is 
an important factor, we re-coded the 
dropout rate as “school without drop- 
out”, a dichotomous variable indicating 
only whether or not a school reported 
any dropouts. Please see appendix E for 
a discussion of the variables used in the 
study. 

8. Charter schools are considered public 
schools in Pennsylvania. 

9. The dropout rate continuous variable 
was converted to the “school without 
dropout” dichotomous variable for the 
purpose of this study, as described in 
appendix E. 
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